Gradient Convergence in Gradient Methods

For the classical gradient method xt+1 = xt − γt∇f(xt) and several deterministic and stochastic variants, we discuss the issue of convergence of the gradient sequence ∇f(xt) and the attendant issue of stationarity of limit points of xt. We assume that ∇f is Lipschitz continuous, and that the stepsize γt diminishes to 0 and satisfies standard stochastic approximation conditions. We show that either f(xt) → −∞ or else f(xt) converges to a finite value and ∇f(xt) → 0 (with probability 1 in the stochastic case). Existing results assume various boundedness conditions such as boundedness from below of f , or boundedness of ∇f(xt), or boundedness of xt. 1 Research supported by NSF under Grant DMI-9625489 2 Dept. of Electrical Engineering and Computer Science, M.I.T., Cambridge, Mass., 02139. 1